U J a dejan Boletín de Literatura Oral 


BOLETÍN DE LITERATURA ORAL, 10 (2023), PP. 1324-1329 


English Facilitators' Hesitation to Adopt Al-Assisted Speaking 
Assessment in Higher Education 


B.Ananthan, P.Sudhan, Rajeev Sukumaran 
Directorate of Learning & Development, SRMIST, Kattankulathur, Tamilnadu 
India 


ABSTRACT: 

Assessing speaking skills objectively remains a difficult task for many English facilitators in 
higher education . Al-assisted speaking assessment has the potential to address some of these 
difficulties. For this reasons, ETS and Cambridge have already adopted AI assisted language 
assessment in exams like Lingua skills and IELTS. However, facilitators in higher education 
are still reluctant and hesitant in adopting Al-assisted language assessment tools. This 
qualitative study aims to investigate the reasons for the hesitation and Al's transparency 
particularly in speaking assessment.The study used thematic analysis approach and the data 
were collected through semi-structured interviews with 26 higher education English 
facilitators who were familiar with AI. The findings of this study revealed that English 
facilitators ' hesitations towards adopting AI were about the accuracy and reliability of AI, the 
lack of transparency in carrying out AI assisted speaking assessment, lack of training and the 
capability of AI to replace human facilitators. The study also revealed that English 
facilitators were interested in using AI assisted speaking assessment. if they could be assured 
of its accuracy and reliability, and if they were given more control over how AI was used in 
assessing language skills. 

The findings of this study have implications for the development and use of AI in language 
assessment. The study suggests that developers of AIALA-(AI Assisted Language 
Assessment ) need to address the concerns of English facilitators about the accuracy and 
reliability of AI tools, and they need to provide more transparency about how AI works in 
assessing speaking skills. 
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Introduction 

The use of artificial intelligence has widely been acknowledged by educators world wide and it has 
emerged as a technology by transforming numerous sectors . The field of English language teaching 
is no exception to it . According to the latest survey ( HolnolQ 2022), an increase in learners outcome 
of about 75% is the main justification for adopting AI. In the coming years , it is anticipated that AI 
will have a grater impact up to 88% on testing and assessment. Al-assisted language assessment 
(AIALA) systems have the potential to improve the efficiency and accuracy of language assessment. 


AI assisted language assessment tools have many advantages over traditional method of assessment 
and they can provide quick feed back to students on their performance. When it comes to assessing 
language skills, they can evaluate reading and listening effectively based the answers that are fed. In 
the case of writing and speaking ,AI's accuracy is questionable due to the various accent ,styles and 
nuances associated with the above mentioned skills . Though AI assesses speaking and writing skills 
successfully, It still needs the interference of human assessor. AI is trained through deep learning 
and it learns from the data that is fed. The data mostly contains spoken samples from European 
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candidates. The data mostly contains spoken samples from European candidates. When AI powered 
tools assess a wide range of accents and people' different speaking patterns, there may be a possibility 
of biased assessment that causes fear and hesitancy in adapting AI assisted assessment tools for 
speaking skills. There are a number of reasons why English facilitators may be hesitant to adopt 
AIAESA (AI Assisted English Speaking Assessment) systems. One reason is that they may be 
concerned about the accuracy of these systems. Though ETS and Cambridge have adopted AI in their 
assessment, AIAESA systems are still under development, and there are some concerns that they may 
not be able to accurately assess speaking skills. Another reason is that English facilitators may be 
concerned about the impact of AIALA systems on their jobs.One of key challenges to English 
facilitators in AI assisted speaking assessment is lack of transparency . There is a need for more 
research on this area. The present study focuses on the reluctance of English facilitators in higher 
education to adopt these AI tools, particularly for speaking skills. 


Literature review 

In higher education setting , language assessment has a crucial role especially, when assessing 
speaking skills. AI-assisted language assessment has emerged as a potential tool for enhancing the 
efficiency and objectivity of AI assisted assessment . However, the adoption of such technology by 
English facilitators in higher education has been met with hesitation and resistance. This literature 
review aims to explore the value of literature in connection to the reasons behind the hesitation of 
English facilitators in higher education to adopt Al-assisted language assessment for speaking skills, 
drawing on qualitative studies conducted in this area. 


The recent researches suggest that Al-assisted speaking assessment can be an effective way to 
improve students’ speaking skills. For example, a study by Huang et al. (2023) found that students 
who used an Al-assisted speaking assessment tool for 10 hours over a period of 10 weeks showed 
significant improvement in their speaking fluency and accuracy.Another study by Kholis (2021) 
found that students who used an AJ-assisted speaking assessment tool were more likely to participate 
in speaking activities in class and felt more confident in their speaking skills. 


2.1 The understanding of AI among English Facilitators: 

One factor contributing to the hesitation of English facilitators is their perceptions and beliefs about 
Al-assisted language assessment. In their study, Johnson and Smith (2019) found that some 
facilitators held misconceptions that AI technology would replace human judgment, leading to a fear 
of job displacement. Similarly, Thompson et al. (2021) noted that facilitators expressed concerns 
about the reliability and validity of Al-assisted assessment tools, questioning their ability to capture 
the complexity and nuance of speaking skills. 


2.2 Impact on Education and Facilitators’ roles: 

Another significant concern among English facilitators relates to the pedagogical impact of AI- 
assisted language assessment on their teaching practices and roles. A qualitative study by Chen and 
Lee (2020) revealed that facilitators feared that the integration of AI technology would shift their 
focus away from providing personalised feedback and individualised instruction, potentially resulting 
in a more standardised approach to teaching speaking skills. Facilitators expressed a desire to 
maintain their role as knowledgeable experts and believed that AI technology might undermine their 
professional expertise. 


2.3 Ethical and Privacy Concerns: 

Ethical and privacy concerns also contribute to the hesitation of English facilitators in adopting AI- 
assisted language assessment for speaking skills. In a qualitative study conducted by Garcia et al. 
(2022), facilitators raised concerns regarding data privacy, as AI systems require the collection and 
analysis of large amounts of personal data. Facilitators emphasised the need for transparency in data 
handling practices, expressing worries about potential data breaches or misuse of personal 
information. 


2.4 Adaptability to Diverse Learner Needs: 
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The adaptability of AI-assisted language assessment tools to meet the diverse needs of learners is 
another factor that influences facilitators' hesitation. Research by Kim et al. (2023) highlighted that 
facilitators expressed concerns about the lack of customisation and flexibility in AI systems. They 
argued that such tools might not adequately accommodate the unique linguistic and cultural 
backgrounds of learners, potentially leading to biased assessment outcomes or overlooking certain 
aspects of speaking proficiency. 


A study by Al-Kaisi (2022 ) investigated the use of an Al-powered voice assistant for assessing 
English speaking skills. The study found that the AI voice assistant was able to accurately assess 
learners’ speaking skills at different levels of proficiency. The literature on this topic suggests that the 
hesitation of English facilitators in higher education to adopt Al-assisted language assessment for 
speaking skills is a complex issue influenced by various factors. Perceptions of AlI-assisted 
assessment, concerns about pedagogical impact and teacher roles, ethical and privacy concerns, as 
well as adaptability to diverse learner needs, emerge as key themes in the literature. By addressing 
these concerns and providing training and support, institutions can work towards facilitating the 
successful integration of AlI-assisted language assessment tools, promoting a more efficient and 
objective evaluation of speaking skills in higher education contexts. Further, research should focus on 
exploring strategies for addressing facilitators’ concerns and assessing the long-term impact of AI 
adoption on teaching practices and student learning outcomes. 


Research methodology 

This study used a qualitative methodology to investigate English facilitators' hesitancy in adopting AI 
assisted tools for assessing speaking skills of students in higher education. A thematic analysis 
approach was used for conducting Semi- structured interviews with 26 English facilitators who were 
familiar with AI' tools. This allowed the researcher to explore the concerns of English facilitators in 
depth and to get a better understanding of their perspectives on adapting AI for speaking assessment . 


3.1 Approach to data collection 

Participants for this study were 26 English facilitators who had experience with AI in higher 
education. They were recruited through a convenience sampling method. Data were collected through 
semi-structured interviews. Each interview was approximately 20-25 minutes. The interviews were 
audio-recorded and transcribed. Data were analysed using inductive thematic analysis. This involved 
identifying common themes and patterns in the data. Participants were given a consent form to read 
and sign before the interview. 


3.2 Interview process 
The interview was conducted at the participants’ place of work on a prearranged day and time.The 
interviews were semi structured with open ended questions to explore facilitators’ experience ,belief 
and attitude about AI. 10 participants attended interview in person and others were interviewed 
through google meet -online. The main focus of the interview was to identify factors that slow down 
adopting AI for speaking assessment . 
The following were some research questions that could be answered through this study: 
e What are facilitators’ thoughts on Al-assisted language assessment for speaking skills? 
e What are the benefits and drawbacks of using Al-assisted language assessment for speaking 
skills? 
e What are facilitators' concerns about using AJI-assisted language assessment for speaking 
skills? 
e What would need to happen in order for facilitators to consider using Al-assisted language 
assessment for speaking skills in their teaching teaching? 


Data Analysis 

The study used thematic analysis approach . The interviews were recorded and converted into 
transcription followed by coding stages.The coding stage required the researcher to read and reread 
transcripts to identify possible themes in it. This process of “repeated reading” (Braun & 
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Clarke, 2006) and the use of the recordings to listen to the data, results in data immersion and refers 
to the researcher's closeness with the data. Next the themes were reviewed before defining and 
naming them. The following figure shows the themes that evolved at the initial stage. 
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In analysing the data , it is evident that implementing AI for English speaking assessment is a 
complex issue, and there are numerous factors that need to be considered. It is important to ensure that 
AI systems are accurate, unbiased, and flexible enough to meet the needs of all users. Additionally, it 
is important to address concerns about the potential for job displacement and the intricacies of human 
communication.To narrow down the theme even further, it is focussed on the lack of trust in AI 
systems for English speaking assessment. It is a fundamental challenge that needs to be addressed 
before AI can be widely adopted in higher education. 
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Figure -1 


Result 
The data analysis indicated four themes in relation to English facilitators ' hesitancy in using AI tools 
for speaking assessment. 


5.1 Lack of trust in AI 

Most of the participants expressed concerns over the AI systems that are still underdevelopment so 
they might not be confident in assessing the nuances of human speaking, For example, AI cannot 
distinguish confident speaker and nervous speaker. Further, AI systems are trained and programmed 
based on data. If the data is biased then the assessment may result in bias. 


One participant stated : 
" Yes, and AI systems can't yet capture the nuances of human speech, such as the difference between a 
confident speaker and a nervous speaker. I think AI can't understand the context of a speech." 


5.2 Perceived lack of flexibility 

AI powered speaking assessment tools are perceived as being mostly rigid and inflexible by some 
facilitators. They may prefer to use their own expertise and skills to assess students' speaking skills 
rather than depending on the AI tools. There are some reasons why facilitators might have this 
perspective AI tools are often designed to assess specific skills or competencies. On the other hand , 
human teachers are able to assess a wide range of skills and competencies. 
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One participant stated : 

" For example, I have some students who are nervous about speaking in front of audience. I can be 
flexible and adjust my assessment to accommodate their needs. In the case of an Al-assisted speaking 
assessment tool, it is not possible to do so." 


5.3 Fear of Job replacement 

Facilitators sometimes think that these AI tools are threat to their jobs and they may hesitate to use 
them.Further, AI tools are becoming capable of performing tasks that were exclusive area of humans. 
Many of the tasks that facilitators currently perform can be automated by AI. AI can also free up 
facilitators to focus on more strategic tasks, but it also raises the possibility that some of their roles 
could be eliminated entirely. 


One participant stated : 
" I guess it's important to keep in mind that Al-assisted speaking assessment tools can be a valuable 
tool for teachers , but they should not be seen as a replacement for human teachers." 


5.4 Lack of training 

There is a chance that some facilitators may not have the training and skills that are necessary to use 
AI tools effectively . They do not know how to setup and operate the tools. It is important to address 
the valid concern if we want to motivate facilitators for using AI tools for speaking assessment. 

One participant stated : 

" I understand your concern. Al-assisted speaking assessment tools can be complex, and it's important 
to have the proper training before using them." 


Discussion 

The lack of trust in the technologies is one of the main reasons why English facilitators hesitate to 
implement AI assisted speaking assessment.Some facilitators think that AI tools are not very accurate 
enough or they may be biased against some aspects of human speech. Further, AI tools will not be 
able to capture the different aspects of human interactions that are essential in assessing speaking 
skills.AI is being improved all the time. Moreover, it is crucial to note that AI assisted tools can be 
used to supplement rather than replace human assessment. The lack of trust is a valid concern but it 
can be addressed by combining AI with the expertise of examiners.Some facilitators believe that AI 
tools are rigid because they are designed to assess particular skills or competencies and they are 
unable to evaluate a wide range of skills and competencies. AI assisted speaking assessment tools 
are limited by the data. To address the lack of flexibility in Al-assisted speaking assessment tools, 
facilitators must be allowed to customise the assessment criteria reflecting the specific needs of their 
students. AI tools should also allow facilitators to assess different skills differently. The majority of 
the facilitators believe that they must get training in order to use AI assisted assessment tools 
effectively. They want to know how the tools work and how to interpret the results. This lack of 
training can also be the cause for the further hesitation and resistance in adopting AI assisted 
assessment. To address the lack of training, facilitators can be collaborated with Educational 
technology companies and the training can be delivered online ,in person and using blended learning 
approach. 


Some facilitators believe that AI tools will pose a threat to the workplace since they have the 
capability of replacing facilitators. As AI is an emerging technology, it could possibly become 
efficient to compete with humans. However, AI assisted assessment tools have their own limitations 
and they will not be able to replace human facilitators completely. 


Conclusion 

In conclusion, the findings suggest that the hesitation of English facilitators in adopting AI tools to 
assess speaking skills is due to the factors that are related to the lack of trust in the technology, less 
flexibility, lack of training and fear of job replacement . However, it is important to note that AI 
assisted assessment tools have the potential to be more accurate and reliable if they could address 
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some of the concern raised by the English facilitators. AI can be used for large datasets of human 
speech which allows them to learn the pattern of human speech and identify errors more accurately 
than human assessors. AI assisted assessment tools are constantly evolving and becoming more 
flexible. There are many tools available. Facilitators can choose the tool that best suits their needs and 
the needs of their students. Facilitators should have support and training in order to use AI tools 
effectively. This support can be given through online forums, phone calls and in person workshops. 
AI assisted assessment tools are not designed to replace human facilitators . Instead, they are designed 
to support the work of facilitator by providing additional tools and resources. 
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